Image processing method, apparatus, and program

ABSTRACT

An image processing method, having the steps of: transforming color image signals representing an original image into a luminance signal and color difference signals; transforming the luminance signal and the color difference signals separately into multi-resolution signals of the level  1  to the level N; applying an inverse multi-resolution transform to the color difference multi-resolution signals, after suppressing the high-frequency components of the level  1  of the color difference multi-resolution signals; applying an inverse multi-resolution transform processing to the luminance multi-resolution signals, after applying a coring processing using a condition for each level different from other levels to the high-frequency signals of each level of the luminance multi-resolution signals; transforming the processed luminance signal and the processed color difference signals into a set of processed color image signals.

TECHNICAL FIELD

[0001] This invention relates to a processing method and processingapparatus of a color image, and in particular, to a processing methodwhich can be applied to an image processing method and an imageprocessing apparatus to be applied to an image which has been made a setof image signals by the scanning of a color photograph.

BACKGROUND ART

[0002] It is put into practice that an image formed on a colorphotographic film is photoelectrically read by a CCD sensor or the like,to be converted into a set of image signals. This set of image signals,after being undergone various kinds of image processing represented bynegative-to-positive reversing, luminance adjustment, color balanceadjustment, granular noise removal, and sharpness enhancement, aredistributed through a medium such as a CD-R, a floppy disk, or a memorycard, or an Internet line, and outputted as a hard copy image by asilver halide photographic paper, an ink jet printer, a thermal printer,or the like, or displayed on a medium such as a CRT, a liquid crystaldisplay, or a plasma display, to be appreciated.

[0003] Generally speaking, an image on a color photographic film isformed of the assembly of dye-clouds of various sizes. For this reason,by the observation of the image enlarged, it is found that mottledgranular unevenness based on the various sizes of dye-cloud are presenton areas which should originally be of a uniform color. As the result ofthis, in an image obtained by photoelectrically reading an image formedon a photographic film, a granular noise signal corresponding to thegranular unevenness is contained. It has been a point of problem thatthis granular noise signal is remarkably strengthened especially with animage processing of sharpness enhancement, which degrades the quality ofthe image.

[0004] For a method of removing a noise included in an image signal, amethod of smoothing by a smoothing filter or a median filter is widelyknown (for example, S. Inoue et al: “Learning Practical Image Processingby C Programming Language” (in Japanese) p. 54, published by Ohm Co.,Ltd.).

[0005] In U.S. Pat. No. 4,812,903, it is proposed a technology forenhancing sharpness while suppressing granularity through it that a setof color image signals are transformed into a luminance signal and colorinformation signals, the luminance signal is further decomposed into alow-frequency component and a high-frequency component, non-linearprocessing is applied to said low-frequency component of the luminancesignal, an enhancement processing is applied to said high-frequencycomponent of the luminance signal, and an adjustment processing isseparately applied to said color information signals; after that, thelow-frequency component of the luminance signal, the high-frequencycomponent of the luminance signal, and the color information signals,which have been processed, are combined to become a set of processedcolor image signals.

[0006] Further, in the publication of the unexamined patent applicationS63-26783, it is proposed a technology for obtaining a naturalenhancement processing with a small change of color tone through it thata set of inputted color image signals are transformed into a luminancesignal and color information signals, spatially wide-ranging informationis abstracted by the application of an averaging filter processing tosaid luminance signal, spatially detailed information is calculated fromthe difference between said luminance signal and said spatiallywide-ranging information, and a specified transform processing isapplied to each of said spatially detailed information and saidspatially wide-ranging information; after that, the spatially detailedinformation, the spatially wide-ranging information, and said colorinformation signals, which have been already processed, are combined tobecome a set of processed color image signals.

[0007] In the publication of the unexamined patent application H9-22460,it is proposed a technology for enhancing sharpness while suppressinggranularity through it that an inputted image signal is decomposed intoa low-frequency component, a medium-frequency component, and ahigh-frequency component, an enhancement processing is applied to saidhigh-frequency component, while suppression processing being applied tosaid medium-frequency component, and the high-frequency component, themedium-frequency component, which have already been processed, and thelow-frequency component are combined to become a processed image signal.Further, also it is proposed in the same publication a technology forsuppressing the roughening of colors through abstracting a luminancecomponent from the aforesaid high-frequency component andmedium-frequency component in process of the above-mentioned processing,and practicing the above-mentioned enhancement/suppression processingand composition on the basis of said luminance component only.

[0008] In the publication of the unexamined patent application2000-215307, it is proposed a technology for suppressing the rougheningresulting from granularity and enhancing sharpness through it that a setof inputted color image signals of R, G, and B are decomposed into alow-frequency component, a medium-frequency component, and ahigh-frequency component, it is obtained a correlation value withrespect to corresponding pixels between at least one color set composedof two colors out of the above-mentioned three colors R, G, and B ofsaid medium-frequency component and/or said high-frequency component, agranularity detecting processing using a specified morphology operationis applied to the above-mentioned R, G, and B signals, and anenhancement processing is applied to said high-frequency component,while a suppression processing is applied to said medium-frequencycomponent on the basis of evaluation values composed of theabove-mentioned correlation value and the result of the granularitydetecting processing using the above-mentioned morphology operation;then, the above-mentioned high-frequency component after the enhancementprocessing, the above-mentioned medium-frequency component after thesuppression processing, and the above-mentioned low-frequency componentare combined to compose a signal. Further, also it is proposed in saidpublication a technology such that, prior to the above-mentionedenhancement processing to the high-frequency component, from theaforesaid medium-frequency component and the aforesaid high-frequencycomponent, a luminance signal concerning said medium-frequency componentand a luminance signal concerning said high-frequency component areabstracted respectively, and the aforesaid enhancement processing to thehigh-frequency component is an enhancement processing to the luminancesignal concerning the aforesaid high-frequency component, while theaforesaid suppression processing to the medium-frequency component is asuppression processing to the luminance signal concerning the aforesaidmedium-frequency component.

[0009] In the publication of the unexamined patent applicationH11-266358, it is proposed a technology in which, in obtaining a set ofprocessed image signals for reproducing a visible image from a set ofdigital original image signals representing a color image, a set of blurimage signals representing a blur image of an original image aregenerated by applying a filtering processing with an edge-preservingsmoothing filter to the above-mentioned set of digital original imagesignals, pixels to become the object of abstracting for the blur imageare abstracted from this set of blur image signals, a specified area ofthe pixels of the object of abstracting of the original imagecorresponding to these pixels is abstracted, and a specified imageprocessing is performed in accordance with this specified area.

[0010] In the publication of the unexamined patent applicationH6-274615, an image processing method to be applied to an image signalrepresenting a radiation image in reproducing a visible image of saidradiation image from said image signal, characterized by comprising thesteps of decomposing said image signal into signals of a plurality offrequency bands by applying a wavelet transform to said image signalwith the second derivative of the smoothing function used as the basicwavelet function, detecting a zero point where the value of said signalbecomes zero in the signal of the lowest frequency band among thesignals falling in the range of the desired frequency range of saidsignals of the plurality of frequency bands, determining an enhancementcoefficient having a larger value in the vicinity of said detected zeropoint than other areas,

[0011] multiplying a signal of the frequency band higher by one stepthan said lowest frequency band by said enhancement coefficient,detecting a zero point where the value of said signal multiplied by saidenhancement coefficient becomes zero, determining an enhancementcoefficient having a larger value in the vicinity of said detected zeropoint than other areas, multiplying a signal of the frequency bandhigher by one step than said frequency band which is higher by one stepthan the lowest frequency band by said enhancement coefficient, carryingout said detection of a zero point, said determination of an enhancementcoefficient, and said multiplication by said enhancement coefficient forall signals falling in the range of said desired frequency band, andapplying an inverse wavelet transform to each of said signals multipliedby said enhancement coefficient and other signals.

[0012] In the publication of the unexamined patent applicationH9-212623, an image processing method characterized by obtaining aprocessed image signal through the steps of decomposing an image signalrepresenting a radiation image into a plurality of frequency band imagesignals representing images of the respective frequency bands bytransforming said image signal into signals of a multi-resolution space,applying a processing to make zero the value of signals having a lowerabsolute value than a specified threshold to specified frequency-bandimage signals out of said plural frequency-band image signals, andapplying an inverse transform to the frequency-band image signals whichhave undergone said processing and other frequency-band signals.

[0013] In the publication of the unexamined patent application2000-224421, it is proposed an image processing method characterized byobtaining a processed image signal through the steps of applying a noiseabstraction processing to a specified frequency-band image signal of aspecified frequency band obtained in the process of decomposing an imagesignal into a plurality of frequency-band signals representing images ofthe respective frequency bands by the application of a multi-resolutiontransform processing to said image signal, obtaining a processedfrequency-band image signal by applying a noise removal processing tosaid specified frequency-band image signal on the basis of the result ofsaid noise abstraction processing, obtaining a frequency-band imagesignal of a frequency band which is lower by one step than saidspecified frequency band by applying a multi-resolution transformprocessing to said processed frequency-band image signal, obtainingprocessed frequency-band image signals of the respective frequency bandsby practicing repeatedly up to the desired frequency band theabove-mentioned noise abstraction processing, the above-mentioned noiseremoval processing, and the above-mentioned multi-resolution transformprocessing using the frequency-band image signal of said frequency bandlower by one step as the above-mentioned specified frequency-band imagesignal, and applying an inverse multi-resolution transform processing tosaid processed frequency-band image signals.

[0014] In the publication of the unexamined patent application2001-189861, it is proposed a method of suppression processing of astill-standing grid for suppressing spatial frequency componentscorresponding to a still-standing grid image for use in an imageprocessing method in which an image having a desired resolution can beobtained by transforming an original image including a still-standinggrid image into an image represented in a multi-resolution space byrepeatedly applying a filtering processing based on a specified filterto said original image, characterized by applying a wavelet transform tosaid original image in said filtering processing of at least the firststage, by means of a low pass filter having a characteristic such thatthe response of spatial frequency components of not less than 97% to thespatial frequency components of the still-standing grid image becomesnot greater than 5%.

[0015] In the publication of the unexamined patent application2001-223899, it is proposed an image processing method using coding anddecoding of an image signal characterized by obtaining a processed imagesignal carrying a processed image through the steps of obtainingmulti-resolution transform signals by the application of amulti-resolution transform processing to an image signal, obtainingprocessed transform signals carrying a processed image to which adesired image processing has been applied by the application of acoefficient transform processing corresponding to said desired imageprocessing to said multi-resolution transform signals, obtainingprocessed coded data carrying said processed image by the application ofa coding processing to said processed transform signals, decoding saidprocessed coded data, and applying an inverse multi-resolution transformprocessing to the decoded data.

[0016] However, because the noise removal based on a simple filterprocessing which is widely known is accompanied by the lowering of imagesharpness, a satisfactory image cannot be obtained. In a technologydisclosed in U.S. Pat. No. 4,812,903 or in a technology disclosed in thepublication of the unexamined patent application S63-26783, althoughsome effect can be recognized in the suppression of granular noise beingmade worse in a sharpness enhancing process, it is not sufficient, andit cannot be expected to suppress a granular noise for a signal beforethe processing.

[0017] The method disclosed in the publication of the unexamined patentapplication H9-22460 has an effect to eliminate granular unevennesslooking mottled in an image by selectively suppressing amedium-frequency component in which a granular noise is mainly present,but because the information on the image structure which is present inthe medium-frequency component is suppressed at the same time, forexample, it tends to happen that the shade in the periphery of thebridge of the nose and that around the eyes are suppressed to give animpression of blurred image of the face. If the high-frequency componentis remarkably enhanced for compensating this impression of blur, theimage becomes unsightly due to the generation of fine noises lookinglike colors being out of registration in the flat areas such as thecheek.

[0018] According to the method proposed also in the publication of theunexamined patent application H9-22460 in which luminance component isabstracted from the high-frequency component and medium-frequencycomponent, and enhancement/suppression processing and combining arepracticed on the basis of said luminance component only, the noiselooking like colors being out of registration is not generated, but asubtle variation in a reddish color of the skin etc. are alsosuppressed, which gives an impression of smooth blank makeup face.

[0019] Further, in any one of the examples of the embodiment describedin the publication of the unexamined patent application H9-22460, if itis attempted to enhance the sharpness in fine areas like hairs, thehigh-frequency component is excessively enhanced, and it is generated anoise looking as if fine powders are scattered over the whole imagearea. Further, there is another problem that, in order to separate alow-frequency component from an image signal as intended in thepublication of the unexamined patent application H9-22460, it isnecessary to apply a processing with a very large-sized low pass filterto every channel of the color signal representing the original image(for example, each of the signal channels of B, G, and R), which makesthe load of calculation for the image processing very heavy; this isalso a very big problem.

[0020] The method proposed in the publication of the unexamined patentapplication 2000-215307 eases somewhat the above-mentioned defect bydynamically controlling the degree of the suppression of themedium-frequency component and the degree of the enhancement of thehigh-frequency component in accordance with the situation, but it cannotbe said that it is sufficient, and it follows the method which has thepoint of problem in the image quality as it is. From the viewpoint ofthe amount of calculation, it is needless to say that this method has aheavier load than that proposed in the publication of the unexaminedpatent application H9-22460.

[0021] The technology disclosed in the publication of the unexaminedpatent application H11-266358 is understood as a technology in which aspecified area of image pixels of the object of abstractingcorresponding to a specified photographic object such as the human skinor the blue sky is fixedly determined by the use of a blur imageproduced by the application of an edge-preserving smoothing filter to anoriginal image, and an image processing is practiced under a specialcondition for the above-mentioned specified area determined. However, aprocessing by means of a large-sized low pass filter is required for theproduction of a blur image, which makes the load of calculation in theimage processing heavy. Further, in the method of abstracting aspecified photographic object using a blur image, it is difficult todiscriminate, for example, “a blue-colored wall from the blue sky”, or“the human skin from a beige-colored wall”, and the accuracy ofdiscrimination is highly insufficient.

[0022] The technology disclosed in the publication of the unexaminedpatent application H6-274615 is construed as an edge enhancingtechnology in which an image signal representing a radiation imageundergoes a multi-resolution transform based on a wavelet transform, anda higher enhancement is applied to a high-frequency signal of the leveln after transform with respect to a point where a high-frequency signalof the level (n+1) becomes zero than other portions. However, when thistechnology is applied to a color image, the RGB balance of aphotographic object near the edge is broken and a false colored contouris produced, which makes the image very unsightly. On top of it, by thetechnology disclosed in the publication of the unexamined patentapplication H6-274615, it is impossible to reduce granular noisesincluded in an image signal.

[0023] The technology disclosed in the unexamined patent applicationH9-212623 is construed as a noise removal technology to carry out whatis called a hard coring processing in which an image signal representinga radiation image undergoes a multi-resolution transform, and out of thehigh-frequency signal values after transform, those signal values notgreater than a threshold is made equally to be zero; however, if saidtechnology is applied to a color image, the RGB balance in the vicinityof the edge of a photographic object is broken to produce a false-colorcontour, which makes the image very unsightly. The noise structure of animage signal obtained by the photoelectric reading of an image formed ona color photographic film by means of a CCD sensor or the like isdifferent from that of a radiation image, and mottled granularunevenness based on the size of color-developing dye-clouds ispredominant in it. If such a large threshold as to dissolve this mottledunevenness is set, the sharpness of the image is remarkably lowered; tostate it in a reverse way, with such a small threshold as to maintainthe sharpness of the image, mottled unevenness is not dissolved. For theabove-mentioned reasons, it is impossible to use the technologydisclosed in the publication of the unexamined patent applicationH9-212623 for a color image.

[0024] The technology disclosed in the publication of the unexaminedpatent application 2000-224421 is construed as a noise removaltechnology based on the repeating of an operation such that, in applyinga multi-resolution transform to an image signal representing an image, anoise abstraction processing is applied to a low-frequency signal afterthe practice of a transform of the nth level, a noise removal processingis applied to said low-frequency signal under a condition based on theresult of said noise abstraction, and after that, a transform of the(n+1)th level is applied to the above-mentioned processed low-frequencysignal. If said technology is applied to a color image, RGB balance isbroken in areas where mottled unevenness is dissolved to producefalse-colored spots, which makes the image very unsightly. Further,every time at the transform of one level, it is necessary to repeat anoise abstraction processing and a noise removal processing, which makesthe load of calculation very heavy.

[0025] Further, in the images for medical use which are supposed in thepublication of the unexamined patent application 2000-224421, kinds ofphotographic object are limited and most of the images are comparativelymonotonous; however on the other hand, it is characteristic of a colorphotographic image that there are present mixedly in the image, areasfor which different image qualities are desired respectively such as anarea where fine structures densely gather against a flat area, a lightarea against a dark area, etc., as observed in a personal portrait witha background of a wood. In an color image like this, it is necessary tochange the condition of noise removal on the basis of the grasping ofthe area structure with a wide view of it; however, in the technologydisclosed in the publication of the unexamined patent application2000-224421, it is necessary to judge the condition of a noise removalprocessing of the nth-level resolution on the basis of the informationof the nth-level resolution concerned; therefore, it is difficult todesign an algorithm for determining the noise removal condition.

[0026] The technology disclosed in the publication of the unexaminedpatent application 2001-189866 can be construed as a technology to use,in applying a multi-resolution transform to a signal of an imageincluding a still-standing grid image, a filter having a characteristicnot to make the main spatial frequency components of a still-standinggrid image substantially pass it in the first-level transform; however,this technology is effective only for the removal of a noise having aperiodic uniform frequency response like a still-standing grid image,and it is not effective for the removal of a noise having a distributionin the shape and size with its spatial frequency components notconcentrated in a particular band like a granular noise in a colorphotograph.

[0027] The technology disclosed in the publication of the unexaminedpatent application 2001-223899 can be construed as an image processingmethod based on the coding and decoding of an image signal such that,after a multi-resolution transform is applied to an image signal and acoefficient transform processing corresponding to the desired imageprocessing to the processed signal, the signal is compressed and codedto produce data which are stored and transferred, and at the time ofdisplay of the image, the data are decoded and undergo an inversemulti-resolution transform. This technology is one to make the speed ofcalculation high by carrying out the image processing and codingprocessing parallel in preparing an image data file premised on the datacompression based on a multi-resolution transform; however, in the fieldof the color photography, it is necessary to prepare a data file basedon a common standard which does not use a multi-resolution transformsuch as a JPEG file or a TIFF file in response to the request of a user,and the method disclosed in the publication of the unexamined patentapplication 2001-223899 cannot make the speed of the calculation high.Further, if the technology of the publication of the unexamined patentapplication 2001-223899 is applied to a color image, it is produced aserious problem also in image quality that a false-color contour isproduced in the vicinity of the edge of a photographic object or thatfalse-color spots are produced in a flat portion.

[0028] It is an object of this invention to provide, on the basis of theabove-mentioned situation, an image processing method and apparatuswhich give only a light load of calculation and are capable ofsuppressing a granular noise signal included in a color image signal aswell as enhancing the sharpness of an image, while preserving the shadein the periphery of the bridge of the nose and that around the eyes,without producing a noise looking like colors being out of registrationand a smooth expressionless makeup face, without producing a noiselooking as if fine powders are scattered, and without producing afalse-color contour in the vicinity of the edge or false-color spots inthe flat portion.

DISCLOSURE OF INVENTION

[0029] An image processing method of the first of this invention ischaracterized by obtaining a processed image signal through the steps ofa set of color image signals representing an original image aretransformed into a luminance signal and color difference signals,transforming said luminance signal and said color difference signalsseparately into multi-resolution signals of the level 1 to the level Nto make them luminance multi-resolution signals and color differencemulti-resolution signals, after suppressing the high-frequencycomponents of the level 1 of said color difference multi-resolutionsignals, applying an inverse multi-resolution transform to said colordifference multi-resolution signals to make them processed colordifference signals, after applying a coring processing using a conditionfor each level different from other levels to the high-frequency signalsof each level of said luminance multi-resolution signals, applying aninverse multi-resolution transform processing to said luminancemulti-resolution signals to make them a processed luminance signal,transforming said processed luminance signal and said processed colordifference signals into a set of processed color image signals.

[0030] An image processing apparatus of the second of this invention isa processing apparatus for practicing an image processing method of thefirst of this invention, and is characterized by comprising means fortransforming a set of color image signals representing an original imageinto a luminance signal and color difference signals, means fortransforming said luminance signal and said color difference signalsseparately into multi-resolution signals of the level 1 to the level Nto make them luminance multi-resolution signals and color differencemulti-resolution signals, a transform means for applying an inversemulti-resolution transform to said color difference multi-resolutionsignals to transform them into processed color difference signals aftersuppressing the high-frequency signals of the level 1 of said colordifference multi-resolution signals, means for applying a coringprocessing using a condition for each level different from other levelsto the high-frequency signals of each level of said luminancemulti-resolution signals, a transform means for applying an inversemulti-resolution transform to the luminance multi-resolution signalshaving already undergone said coring processing to transform them into aprocessed luminance signal, and a transform means for transforming saidprocessed luminance signal and said processed color difference signalsinto a set of processed color image signals.

[0031] A program of the third of this invention is a program for makinga computer practice an image processing method of the first of thisinvention, and is one to make a computer function as means fortransforming a set of color image signals representing an original imageinto a luminance signal and color difference signals, means fortransforming said luminance signal and said color difference signalsseparately into multi-resolution signals of the level 1 to the level Nto make them luminance multi-resolution signals and color differencemulti-resolution signals, a transform means for applying an inversemulti-resolution transform to said color difference multi-resolutionsignals to transform them into processed color difference signals aftersuppressing the high-frequency signals of the level 1 of said colordifference multi-resolution signals, means for applying a coringprocessing using a condition for each level different from other levelsto the high-frequency signals of each level of said luminancemulti-resolution signals, a transform means for applying an inversemulti-resolution transform to the luminance multi-resolution signalshaving already undergone said coring processing to transform them into aprocessed luminance signal, and a transform means for transferring saidprocessed luminance signal and said processed color difference signalsinto a set of processed color image signals.

[0032] In this invention, the term “to transform a set of color imagesignals into a luminance signal and color difference signals” means, forexample, to transform a set of three color signals of B, G, and R of anoriginal image into a set of signals of YIQ basis, HSV basis, YUV basis,etc. publicly known to persons specialized in the art, or to transformthem into a set of signals of XYZ basis of CIE 1931 standardcalorimetric system on the basis of the standard such as sRGB or NTSC,or into a set of signals of L★a★b★ basis, or L★u★v★ basis recommended byCIE 1976. Even though the transform into a luminance signal and colordifference signals in this invention is not a perfectly stricttransform, it exhibits a sufficient effect; therefore, it includes as anexample of the embodiment, for example, a transform as described in anexample of practice in the publication of the unexamined patentapplication S63-26783 such that the average value of B, G, and R signalsare made to be the luminance signal and signals in respect of the twoaxes orthogonal to this are made to be the color difference signals.

[0033] Further, the term “a multi-resolution transform method” is ageneral name of a method represented by a wavelet transform method, aperfect-restructure filter bank method, a Laplacian pyramid method,etc., and is one to obtain multi-resolution signals by carrying out theseparation of an input signal into a low-frequency component and ahigh-frequency component and a down-sampling (thinning out of pixels)through a transform operation of one time, and repeating the sameoperation for the low-frequency component obtained. Further, in the casewhere the multi-resolution signals obtained are subjected as it is to aninverse multi-resolution transform without undergoing any processing,the original input signal is completely restructured. This method isexplained in detail, for example, in “Wavelet Analysis and Filter Bank”(authored by G. Strang and T. Nguyen, published by Baifukan Co., Ltd.).

[0034] The number of times of repeating the transform operation in theabove-mentioned multi-resolution transform is called a level. As regardsthe way of naming the level, although there are some differences betweenpersons specialized in the art, in this invention, the way of namingsuch that a signal of higher resolution is made one of the lower levelis employed. That is, in the expression in this invention, the levelzero represents an input signal, and the result of the application of atransform operation once to this is named the level 1, and the result ofthe application of the second transform operation to this result isnamed the level 2.

[0035] This invention does not limit the method of the multi-resolutiontransform to a certain one, but it is particularly desirable from theviewpoint of calculation efficiency to use a wavelet transform. The modeof a transform of one level in the case where a wavelet transform isused as a multi-resolution transform will be explained in more detailwith reference to FIG. 1.

[0036] By applying a one-dimensional low pass filter LPF in the xdirection to an input signal S_(n), and further, thinning out the resultof it by removing one out of every two by a down-sampling means 2⇓, alow-frequency component SX_(n+1) which has a half resolution in the xdirection to the input signal is obtained. Further, by applying aone-dimensional high pass filter HPF in the x direction to the inputsignal S_(n), and further, thinning out the result of it by removing oneout of every two by a down-sampling means 2⇓, a high-frequency componentWX_(n−1) which has a half resolution in the x direction to the inputsignal is obtained.

[0037] Next, by applying a one-dimensional low pass filter LPF in the ydirection to the above-mentioned low-frequency component SX_(n+1), andfurther, thinning out the result of it by removing one out of every twoby a down-sampling means 2⇓, a low-frequency component S_(n+1) which hasa half resolution in the x direction and y direction to the input signalis obtained. In the same way, by applying a one-dimensional high passfilter HPF in the y direction to the low-frequency component S_(n+1),and further, thinning out the result of it by removing one out of everytwo by a down-sampling means 2⇓, a high-frequency component Wh_(n+1)which has a half resolution in the x and y directions to the inputsignal is obtained. By applying the same operations also to theabove-mentioned high-frequency component WX_(n+1), high-frequencycomponents Wv_(n+1) and Wd_(n+1) which have a half resolution in the xand y directions are obtained. By the above-mentioned operations, theinput signal S_(n) is transformed into 4 components S_(n+1), Wh_(n+1),Wv_(n+1), and Wd_(n+1) having a half resolution in the x and ydirections. The above-mentioned transform corresponds to amulti-resolution transform of one level.

[0038] In this invention, a low-frequency component of the level (n+1)means the above-mentioned S_(n+1), and high-frequency components of thelevel (n+1) mean the 3 components of Wh_(n+1), Wv_(n+1), and Wd_(n+1).FIG. 2 is a drawing explaining the mode of an inverse transform of 1level in the case where an inverse wavelet transform is used as aninverse multi-resolution transform. After the insertion of zero at everyother point in the y direction by an up-sampling means 2↑, the inputsignal S_(n+1) is smoothed by a low pass filter LPF′, and after theinsertion of zero at every other point in the y direction by anup-sampling means 2↑, the input signal Wh_(n+1) is processed by a highpass filter HPF′, and by the addition of the two results, SX_(n+1) isobtained. By the application of the same processing to the input signalsWv_(n+1) and Wd_(n+1) too, WX_(n+1) is obtained. Further, by theapplication of the same processing to the above-mentioned SX_(n+1) andWX_(n+1), S_(n) having a twice resolution in the x and y directions isoutputted.

[0039] In the case where the filters LPF and HPF to be used in theabove-mentioned wavelet transform in FIG. 1 are the same as the filtersLPF′ and HPF′ to be used in the above-mentioned inverse wavelettransform in FIG. 2, the transform is called an orthogonal wavelettransform, and in the case where the filters used in both transformprocesses are different, the transform is called a biorthogonal wavelettransform. In this invention, a biorthogonal wavelet transform in whichthe coefficients of these filters are laterally symmetrical with respectto the center line is more desirable. To state it in more detail, abiorthogonal wavelet transform in which the length of each filtercoefficient is from 3 to 13 is used particularly desirably. An actualexample of desirable coefficients are shown below.

EXAMPLE 1 Cohen, Daubechies, Feauveau 9-7

[0040] TABLE 1 for Multi-resolution for Inverse multi-resolutiontransformation transformation Coefficient of Coefficient of Coefficientof Coefficient of low pass high pass low pass high pass filter filterfilter filter 0.037829 −0.037829 −0.023849 −0.064539 −0.064539 −0.023849−0.110624 0.04069 −0.04069 0.110624 0.377403 0.418092 0.418092 0.3774030.852699 −0.788485 0.788485 −0.852699 0.377403 0.418092 0.4180920.377403 −0.110624 0.04069 −0.04069 0.110624 −0.023849 −0.064539−0.064539 −0.023849 0.037829 −0.037829

EXAMPLE 2 Cohen, Daubechies, Feauveau 5-3

[0041] TABLE 2 for Multi-resolution for Inverse multi-resolutiontransformation transformation Coefficient of Coefficient of Coefficientof Coefficient of low pass high pass low pass high pass filter filterfilter filter −0.176777 0.176777 0.353553 0.353553 0.353553 0.3535531.06066 −0.707107 0.707107 −1.06066 0.353553 0.353553 0.353553 0.353553−0.176777 0.176777

EXAMPLE 3 Spline 4-12

[0042] TABLE 3 for Multi-resolution for Inverse multi-resolutiontransformation transformation Coefficient of Coefficient of Coefficientof Coefficient of low pass high pass low pass high pass filter filterfilter filter −0.013811 0.013811 0.041432 0.041432 0.052481 −0.052481−0.267927 −0.267927 −0.071816 0.176777 0.176777 0.071816 0.966748−0.53033 0.53033 0.966748 0.966748 0.53033 0.53033 −0.966748 −0.071816−0.176777 0.176777 −0.071816 −0.267927 0.267927 0.052481 0.0524810.041432 −0.041432 −0.013811 −0.013811

[0043]FIG. 3 is a drawing showing the concept of a multi-resolutiontransform and an inverse multi-resolution transform in multiple levels.An input signal S₀ is decomposed into high-frequency components of Nlevels and a low-frequency component of the level N by transformoperations of N times. After an image processing to be described lateris applied to every component produced, by the inverse transformoperations of N times to every component, a processed signal S₀′ isoutputted. The number of times of practicing transform operationsdepends on the size, resolution, and the strength of the noise of theinput signal S₀, and the operations of 2 to 20 times are desirable andto state it in particular, operations of 2 to 8 times are moredesirable.

[0044] In the case where a one-dimensional filter bank method or aLaplacian pyramid method is employed for the multi-resolution method ofthis invention, the same calculation procedure can be applied exceptthat a difference is produced in the processing by the down-samplingmeans 2⇓ or the up-sampling means 2↑ in the transform or in the inversetransform shown in FIG. 1 and FIG. 2 in which the processing is notlimited to every other one, and the concept of the multi-resolutiontransform and the inverse multi-resolution transform shown in FIG. 3 canbe commonly used. In the case where a transform based on atwo-dimensional Laplacian pyramid is used for the multi-resolutiontransform of this invention, the same concept shown in FIG. 3 can beapplied except that there is a difference in the number of thehigh-frequency component obtained by the transform of one time, whichbecomes 1 in this case.

[0045] In this invention, the term “after suppressing the high-frequencycomponents of the level 1 of said color difference multi-resolutionsignals, applying an inverse multi-resolution transform to said colordifference multi-resolution signals to make them processed colordifference signals” means, for example, an operation such that, after aset of color input signals are transformed into a luminance signal Y, acolor difference signal I, and a color difference signal Q of YIQ basis,a multi-resolution transform shown in FIG. 3 is applied to each of thecolor difference signals I and Q as the input signal S₀, a suppressionsignal processing is applied to the high-frequency components Wh₁, Wv₁,and Wd₁ abstracted by the first transform, and S₀′ which has beenrestructured by the use of the high-frequency components Wh₁′, Wv₁′, andWd₁′ obtained after suppression in an inverse multi-resolution transformis made to be a processed color difference signal I′ and a processedcolor difference signal Q′.

[0046] As regards the suppression signal processing to be applied tothese color difference signals, there is a method to make the signalvalues equally zero, a method to multiply the signal values by apositive constant smaller than 1, or the like. Further, the judgementwhether or not a suppression signal processing is to be applied to thehigh-frequency components Wh_(n), Wv_(n), and Wd_(n) (1<n≦N) obtained bythe multi-resolution transform operations on and after the second onedepends on the resolution and the condition of the noise of the imagesignal representing an original image. In this invention, it is adesirable mode of practice to suppress only the high-frequency signalsof the level 1 of the color difference multi-resolution signals or tosuppress only the high-frequency signals of the level 1 and level 2 ofthe color difference multi-resolution signals. In this case, for thenumber of times N of the multi-resolution transform operations to beapplied to the color difference signals, 1 or 2 is sufficient.

[0047] In this invention, the term “coring processing” means a transformprocessing to apply suppression to an input signal in accordance withits strength, and for a simplest example, a method to make a portionwhere the absolute value is smaller than a threshold to be zero can becited. An example is shown in FIG. 4. In a hard coring processing, aportion of an input signal where its absolute values are not greaterthan a threshold is equally outputted as zero, and if an absolute valueof an input signal is greater than the threshold, it is outputted withits value kept as it is (FIG. 4(a)). In a soft coring processing, aportion of an input signal where its absolute values are not greaterthan a threshold is equally outputted as zero, and if an absolute valueof an input signal is greater than the threshold, it is outputted withits absolute value subtracted by the threshold (FIG. 4(b)). Also asregards a non-linear coring processing, it is the same in the point thata portion of an input signal where the absolute values are not greaterthan a threshold is equally made to be zero, but in the case where anabsolute value of an input signal is greater than a threshold, thedegree of suppression is varied with a function of the absolute value ofthe input signal (FIG. 4(c)). The detail of a non-linear coringprocessing will be explained in more detail later.

[0048] In this invention, the term “after applying a coring processingusing a condition for each level different from other levels to thehigh-frequency signals of each level of said luminance multi-resolutionsignals, applying an inverse multi-resolution transform processing tosaid luminance multi-resolution signals to make them a processedluminance signal,” means, for example, an operation after a set of colorinput signals are transformed into a luminance signal Y, a colordifference signal I, and a color difference signal Q of YIQ basis, and amulti-resolution transform shown in FIG. 3 is applied to the luminancesignal Y taken as the input signal S₀, applying the above-mentionedcoring processing to the high-frequency components Wh_(n), Wv_(n), andWd_(n) (1≦n≦N) obtained under a condition for each level different fromother levels, and then restructuring S₀′ by the use of Wh_(n)′, Wv_(n)′,and Wd_(n)′ (1≦n≦N) having already undergone the coring processing in aninverse multi-resolution transform to make it a processed luminancesignal Y′. In addition, the number of times of the multi-resolutiontransform operations N to be applied to the luminance signal Y dependson the resolution and the condition of the noise of the set of imagesignals representing an original image, and normally it is desirable tomake it not smaller than 2 and not greater than 6.

[0049] Further, the term “a coring processing using a condition for eachlevel different from other levels” means to change the threshold of asoft coring processing or a hard coring processing or to vary the shapeof the transform curve of a non-linear coring processing, in accordancewith the level number n of the high-frequency components Wh_(n), Wv_(n),and Wd_(n) (1<n≦N). In expressing the threshold for each level or theshape of the transform curve, there is a noteworthy item. As regards thedetermination of the coefficients of a low pass filter for use in amulti-resolution transform, two kinds of method, that is, a method inwhich the total value is normalized to be the square root of 2, and amethod in which the total value is normalized to be 1 are generallyemployed by persons specialized in the art. In the case where the formermethod in which the normalization is made to the square root of 2, asthe result of filter processing being carried out two times in the xdirection and in the y direction for each transform of one level, thesignal values are enlarged twice for each transform operation of onelevel. That is, the result of division operation such that a frequencycomponent of the level n obtained by a multi-resolution transform usinga filter normalized in such a way as to make the total value the squareroot of 2 is divided by 2^(n) becomes equivalent to the result of thelevel n by a multi-resolution transform using a filter normalized insuch a way as to make the total value 1.

[0050] In this invention, the term “a coring processing using acondition for each level different from other levels” does not state theabove-mentioned difference of signal strength for each level due to thenormalization of the filter coefficients, but it states that the coringcondition for each level is different from other levels in thecomparison such that the result of the transform is made to becomeequivalent to the result of a multi-resolution transform using a filternormalized in such a way as to make the total value 1.

[0051] To state it in more detail, for the coring condition of the highfrequency components of the level 1, it is desirable a condition suchthat the value of pixels of an amount of not less than 5% and not morethan 50% to the signal pixels is substantially suppressed to zero. Asregards the coring condition of the high-frequency components of thelevel 2, it is desirable a condition such that the proportion of thepixels whose value is substantially suppressed to zero is equal to orless than that in the case of level 1. As regards the coring conditionof the high-frequency components of the level n (n>2), it is desirable acondition such that the proportion of the pixels whose value issubstantially suppressed to zero is not greater than 20% and less thanthe proportion of the pixels whose value is substantially suppressed tozero in the level (n−1).

[0052] Further, for the three high-frequency components Wh_(n), Wv_(n),and Wd_(n) belonging to the same level, it is appropriate to carry out acoring processing using the same condition; however, it is moredesirable to carry out a coring processing under a condition such thatfor the Wd_(n) which is of high-frequency in both directions of x and y,the proportion of the pixels whose value is substantially suppressed tozero is not less than 1.1 times and not more than 2.0 times of theproportion of the pixels whose value is substantially suppressed to zeroin respect of the remaining high-frequency components Wh_(n) and Wv_(n).

[0053] An image processing method of the fourth of this invention ischaracterized by the obtaining of a processed image signal through thesteps of transforming a set of color image signals representing anoriginal image into a luminance signal and color difference signals,transforming said luminance signal and said color difference signalsseparately into multi-resolution signals of the level 1 to the level Nto make them luminance multi-resolution signals and color differencemulti-resolution signals, after suppressing the high-frequencycomponents of the level 1 of said color difference multi-resolutionsignals, applying an inverse multi-resolution transform to said colordifference multi-resolution signals to make them processed colordifference signals, after applying a coring processing using a conditionfor each level different from other levels to the high-frequency signalsof each level of said luminance multi-resolution signals, applying aninverse multi-resolution transform processing to said luminancemulti-resolution signals to make them a processed luminance signal,applying a sharpness enhancement processing to said processed luminancesignal to make it a luminance signal of enhanced sharpness, transformingsaid luminance signal of enhanced sharpness and said processed colordifference signals into a set of processed image signals.

[0054] An image processing apparatus of the fifth of this invention is aprocessing apparatus for practicing an image processing method of thefourth of this invention, and is characterized by comprising means fortransforming a set of color image signals representing an original imageinto a luminance signal and color difference signals, means fortransforming said luminance signal and said color difference signalsseparately into multi-resolution signals of the level 1 to the level Nto make them luminance multi-resolution signals and color differencemulti-resolution signals, a transform means for applying an inversemulti-resolution transform to said color difference multi-resolutionsignals to transform them into processed color difference signals aftersuppressing the high-frequency signals of the level 1 of said colordifference multi-resolution signals, means for applying coringprocessing using a condition for each level different from other levelsto the high-frequency signals of each level of said luminancemulti-resolution signals, a transform means for applying an inversemulti-resolution transform to the luminance multi-resolution signalshaving already undergone said coring processing to transform them into aprocessed luminance signal, a sharpness enhancing means for applying asharpness enhancement processing to said processed luminance signal tomake it a luminance signal of enhanced sharpness, and a transform meansfor transforming said luminance signal of enhanced sharpness and saidprocessed color difference signals into a set of processed color imagesignals.

[0055] A program of the sixth of this invention is a program for makinga computer practice an image processing method of the fourth of thisinvention, and is one to make a computer function as means fortransforming a set of color image signals representing an original imageinto a luminance signal and color difference signals, means fortransforming said luminance signal and said color difference signalsseparately into multi-resolution signals of the level 1 to the level Nto make them luminance multi-resolution signals and color differencemulti-resolution signals, a transform means for applying an inversemulti-resolution transform to said color difference multi-resolutionsignals to transform them into processed color difference signals aftersuppressing the high-frequency signals of the level 1 of said colordifference multi-resolution signals, means for applying coringprocessing using a condition for each level different from other levelsto the high-frequency signals of each level of said luminancemulti-resolution signals, a transform means for applying an inversemulti-resolution transform to the luminance multi-resolution signalshaving already undergone said coring processing to transform them into aprocessed luminance signal, a sharpness enhancing means for applying asharpness enhancement processing to said processed luminance signal tomake it a luminance signal of enhanced sharpness, and a transform meansfor transforming said processed luminance signal and said processedcolor difference signals into a set of processed color image signals.

[0056] In this invention, the term “applying a sharpness enhancingprocessing to a processed luminance signal” means applying a method inwhich a high pass filter such as a Laplacian filter, a Sobel filter, ora Hueckel filter known to the public is applied to a processed signalrestructured by an inverse multi-resolution transform to abstract itsedge component and it is added to the signal, or applying a method ofsharpness enhancing processing using an unsharp mask to a processedluminance signal.

[0057] The technology concerning such a sharpness enhancement processingis explained, for example, in “Learning Practical Image Processing by CProgramming Language (in Japanese) (authored by S. Inoue et al,published by Ohm Co., Ltd.) in detail. Among these methods, a methodusing a two-dimensional filter with its filter coefficients arrangedsymmetrically with respect to the center is especially desirable. A moredesirable method is such one that a low pass filter with its coefficientof the central cell made maximum and the coefficient of the surroundingcells decreased gradually in accordance with the below describedGaussian function is applied to a processed luminance signal which hasbeen restructured by an inverse multi-resolution transform to abstract alow-frequency component, and a luminance signal of enhanced sharpness isobtained by adding a product of a high-frequency component which hasbeen obtained from the difference between the original processedluminance signal, which has been restructured by an inversemulti-resolution transform, and said low-frequency component multipliedby a number larger than 1 to said low-frequency component. An example ofpractice of this processing is shown in FIG. 5.

[0058] A Gaussian function: f(x)=exp(−x²/2σ²),

[0059] x: distance from the center, σ: standard deviation.

[0060] It is desirable that a low pass filter for use in theabove-mentioned sharpness enhancement processing is a regularsquare-shaped filter with its one side composed of odd number of pixelsof 3 to 9.

[0061] In this invention, it is desirable that a set of image signalsrepresenting an original image are digital image signals obtained by thescanning of a dye image formed on a silver halide film.

[0062] A silver halide film designates a color negative film or a colorreversal film, and a dye image of a color negative film or a colorreversal film is photoelectrically converted into a set of transmissionlight quantity signals by means of a film scanning apparatus known tothe public with a light receiving element such as a line CCD sensor oran area CCD sensor. The set of transmission light quantity signalsobtained are amplified by an amplifier, and are converted into a set ofdigital signals by an A/D conversion device. Subsequently, correctionsfor removing noises proper to a light receiving element such as a darkfixed pattern noise correction and a shading correction are practiced,and further, a calibration processing for correcting the individualdifference of the apparatus caused by the dispersion of thecharacteristics of a light receiving sensor, a color separation filter,a light source lamp, a lens, and other optical parts is carried out.Subsequently, the corrected set of transmission light quantity signalsare transformed into a set of density signals by a logarithmic transformor the like. This set of density signals are sent to a processingcondition judgement section, where processing conditions for practicingimage processing is calculated, and on the basis of these processingconditions, the above-mentioned set of density signals are transformedinto a set of color image signals which have already undergone a colorbalance correction processing, a gradation adjustment processing, anegative-to-positive reversing processing.

[0063] It is desirable to use the set of color image signals which havealready undergone a color balance correction processing, a gradationadjustment processing, a negative-to-positive reversing processing as aset of image color image signals representing an original image of thisinvention. Further, it may be appropriate to practice the process forobtaining a set of color image signals representing an original imageand the process of the image processing of this invention in one and thesame apparatus having the both functions, or also it is appropriate topractice the former process and the latter process in differentapparatus respectively. In the case where either of the above-mentionedprocesses are practiced in different apparatus, a set of color imagesignals representing an original image are transmitted to apparatus forpracticing this invention through a communication line or a medium suchas a CD-ROM. It is appropriate that the mode of the transmission data inthis case is made to follow a format of an image file known to thepublic, and it is desirable that the format is a compression orreversible compression format and the number of quantization bits islarger than the number of quantization bits of an image file to beoutputted by an apparatus for practicing this invention.

[0064] In an image processing method of the first and fourth of thisinvention, it is desirable that the aforesaid means for applying acoring processing to the luminance multi-resolution signals includes atleast one processing to be practiced under a non-linear coring conditionsuch that the rate of change for the signal values of higher rankcorresponding to at least 5% to the total number of signal values withrespect to the absolute value of an input signal becomes smaller than10%, and the signal values of lower rank corresponding to at least 15%to the total number become substantially zero.

[0065] Further, in an image processing apparatus of the second and fifthof this invention, it is desirable that the aforesaid means for applyinga coring processing to the luminance multi-resolution signals includesat least one means for practicing a processing under a non-linear coringcondition such that the ratio of change for the signal values of higherrank corresponding to at least 5% to the total number of signal valueswith respect to the absolute value of an input signal becomes smallerthan 10%, and the signal values of lower rank corresponding to at least15% to the total number become substantially zero.

[0066] Further, in a program of the third and sixth of this invention,it is desirable that the aforesaid means for applying a coringprocessing to the luminance multi-resolution signals includes at leastone means for practicing a processing under a non-linear coringcondition such that the ratio of change for the signal values of higherrank corresponding to at least 5% to the total number of signal valueswith respect to the absolute value of an input signal becomes smallerthan 10%, and the signal values of lower rank corresponding to at least15% to the total number become substantially zero.

[0067] In the above-mentioned description, the term “an input signal”means an input signal to which a coring processing is to be applied, andto state it concretely, it corresponds to the high-frequency componentsWh_(n), Wn_(n), and Wd_(n) (1≦n≦N) in the above-mentioned explanation.The term “the signal values of higher-rank corresponding to at least 5%to the total number of signal values with respect to the absolute valueof an input signal” means the signal values of higher rank fallingwithin a range of at least 5% to the total number of signal values whenthe total signal values of one component arranged in the order of theirabsolute values from the largest to the smaller, and for example, whenthe coring condition of the Wh₃ is discussed, the comparison is madebetween the signal values belonging to the Wh₃ only (In this case, Wh₂or Wv₃ does not become the object of the comparison, for example.).Further, the term “the signal values become substantially zero” meansthat the signal values after a coring processing become smaller than thevalue of the signal corresponding to a position of 4% to the totalnumber of signal values from the smallest in the case where the signalvalues of an input signal are compared as arranged in the order of theabsolute value from the largest to the smaller. It is desirable that thecoring condition specified here is applied to the coring of the level 1and/or the level 2.

[0068] In this invention, it is desirable to change the condition of thecoring processing of the luminance multi-resolution signals of the leveln with reference to a high-frequency signal of a higher level than n.

[0069] The content of the above-mentioned will be explained in moredetail. As regards a high-frequency component obtained by amulti-resolution transform, the higher the number of the level nbecomes, the presence of the high-frequency component of lowerresolution, that is, the wider-range edge component it expresses.Further, the higher the number of level becomes, the more the number ofpixels decreases in proportion to the resolution. For example, in thecase where a wavelet transform is used for the multi-resolutiontransform, the number of pixels of one side of a component of the level1 is ½ of the number of pixels of one side of an image signalrepresenting an original image, and the number of pixels of one side ofa component of the level 2 is ½ of the number of pixels of one side of acomponent of the level 1. Hence, the information at the coordinates (x,y) of a high-frequency component of the level n corresponds to that atthe coordinates (x/2^(i), y/2^(i)) of a high-frequency component of thelevel (n+i). Also in the case where a filter bank method or a Laplacianpyramid method is used for the multi-resolution transform, arelationship corresponding to the above-mentioned is effective on thebasis of the down-sampling condition.

[0070] The term “to change the condition of the coring processing of theluminance multi-resolution signals of the level n with reference to ahigh-frequency signal of a level higher than n” means that, in applyinga coring processing to the signal value at the coordinates (x, y) of thehigh-frequency component Wd_(n) of the level n for example, thethreshold or the transform curve is selected in accordance with thesignal value at the coordinates (x/2^(i), y/2^(i)) of the correspondinghigh-frequency component Wd_(n+i) of the level (n+i). In the above, inthe case where the signal value at the coordinates (x/2¹, y/2^(i)) ofthe high-frequency component Wd_(n+i) of the higher level is large, itis high the possibility that the coordinates (x, y) of the object ofcoring Wd_(n) correspond to an edge area with a wide view of it taken,and because it is desirable not to suppress the signal value of ahigh-frequency component at coordinates corresponding to an edge area ofa photographic object, the condition of coring is eased. On thecontrary, in the case where the signal value at the coordinates(x/2^(i), y/2^(i)) of the high-frequency component Wd_(n+i) of thehigher level is small, it is high the possibility that the coordinates(x, y) of the object of coring Wd_(n) corresponds to a flat area with awide view of it taken, and because a granular noise in a flat area isnoticeable, the condition of coring is strengthened. For thehigh-frequency signal of a high level to be referred to for the purposeof the above-mentioned judgement, it is desirable to use ahigh-frequency signal of a level higher than that of the high-frequencysignal of the object of coring by 1 to 3 levels. That is, in theabove-mentioned example, it is desirable to make i satisfy theinequality 1≦i≦3. It is possible that the number of levels to bereferred to is 1, but also it is appropriate to practice a judgementwith reference to a plurality of levels and on the basis of a weightedaverage of the plural values referred to.

[0071] Further, in this invention, it is desirable that the condition ofthe coring processing of a luminance multi-resolution signal of thelevel n is such one as to preserve the low-frequency signal of eachlevel generated in a multi-resolution transform and to be changed withreference to the low-frequency signal of the level n.

[0072] The above-mentioned will be explained in more detail. Generallyspeaking, the range of the luminance distribution of an object scene ina photographic image extends over a range of 10³ or 10⁴ in an arbitraryunit, but the luminance range which can be displayed by a display deviceor a print is only an order of 10². Therefore, because the wideluminance distribution of a scene of a photographic object is compressedto a luminance range which can be displayed in producing an imagesignal, an operation to compress the gradation in the both sides of highluminance and low luminance is practiced in most cases. For the imagesignal representing an original image in this invention, there is thepossibility that an image signal after the application of theabove-mentioned compression processing is used. In this case, becausethe gradation in the both sides of high luminance and low luminance iscompressed, if high-frequency components in these regions are too muchsuppressed, a risk to destroy the depiction of the fine structures of aphotographic object is raised. On the other hand, because the gradationin the medium luminance region is not compressed, if the suppression ofthe high-frequency components is insufficient, a granular noise becomesnoticeable.

[0073] Further, in the case where an image signal representing anoriginal image is obtained by the scanning of a silver halide film,caused by the characteristic of a silver halide film such that thegranular noise becomes stronger in the darker areas, graininess in theshadow area becomes noticeable. However, in such a strong granularityprocessing condition as to dissolve the granularity in a shadow areacompletely, the fine structure in the highlight area is lost, whichmakes the image give an impression of blur. In view of these situations,this invention provides a technology to make it possible to dynamicallyadjust the condition of removing a granular noise in accordance with theluminance level of an image.

[0074] Generally speaking, in the inverse multi-resolution transform ofa signal which has undergone multi-resolution transforms up to the levelN, if the low-frequency component of the level N and the high-frequencycomponents of the levels 1 to N are given, the original signal can berestructured. In short, the low-frequency components of the levels 1 to(N−1) are unnecessary. Therefore, in the calculation of a usualmulti-resolution transform, the low-frequency components of the levels 1to (N−1) are abandoned in most cases from the viewpoint of the saving ofthe storage capacity.

[0075] In this invention, the term “to preserve the low-frequency signalof each level generated in a multi-resolution transform” means topreserve the low-frequency components of the levels 1 to (N−1)calculated in the process of calculation in a form to be used for thereference to be described later without abandoning them.

[0076] Further, the term “the condition of the coring processing of aluminance multi-resolution signal of the level n is changed withreference to the low-frequency signal of the level n” means, forexample, in applying a coring processing to the signal value at thecoordinates (x, y) of the high-frequency component Wd_(n) of the leveln, the threshold value or the transform curve is selected in accordancewith the signal value at the coordinates (x, y) of the preservedlow-frequency component S_(n). By this means, it becomes possible todynamically adjust the condition of removing a granular noise inaccordance with the luminance level of an image.

[0077] Further, in this invention, it is desirable that the condition ofthe coring processing of a luminance multi-resolution signal of thelevel n is such one as to preserve the low-frequency signal of eachlevel generated in the multi-resolution transform, and to be changedwith reference to a low-frequency signal of a level higher than n.

[0078] The above-mentioned will be explained in more detail. In the casewhere a fine density variation exhibited in a specified range ofneighborhood as viewed widely is recognized as one not expressing thestructure of a photographic object, the fine density variation isrecognized as a granular noise. That is, a granular noise is one not tobe generated by the signal value of one pixel, but one to be generatedby the relationship between the signal values of the neighboring pixels.Hence, in the case where the condition of removing a granular noise isadjusted in accordance with the luminance level of an image, as regardsthe luminance level of an image for use in a condition adjustment, it isdesirable to make the judgement by not only the luminance value of thepixel of the object of noise removal but also the widely-viewedluminance value with the luminance value of the neighboring pixels takeninto account.

[0079] It will be explained a case taken for instance where it is usedas a multi-resolution transform, a transform in which the number ofpixels in the x direction and y direction is made ½ for each one-leveltransform in the same way as a wavelet transform. In order to determinethe coring processing condition of the signal value at the coordinates(x, y) of the high-frequency component of the level n Wd_(n), if thesignal value at the coordinates (x/2^(i), y/2^(i)) of the low-frequencycomponent of the level (n+i) S_(n+i) is referred to, the referencesignal value indicates the average luminance value over (2^(i)×2^(i))pixels in the neighborhood of the coordinates (x, y) of thehigh-frequency component of the level n Wd_(n) which is the object ofthe coring processing. Hence, it is possible to determine a coringprocessing condition on the basis of a widely-viewed luminance valuehaving the luminance value of the neighboring pixels taken into accountby this invention.

[0080] For the low-frequency signal of a level higher than n to bereferred to for the purpose of the above-mentioned judgement, it isdesirable to use a low-frequency signal of a level higher than that ofthe high-frequency signal by 1 to 3 levels. That is, in theabove-mentioned example, it is desirable to make i satisfy theinequality 1≦i≦3. The number of levels to be referred to may be one, butalso it is appropriate to practice the judgement with reference tosignals of a plurality of levels and on the basis of the weightedaverage of the signal values for example.

[0081] In an image processing method of the first and fourth of thisinvention, it is desirable that the low-frequency signals of each levelgenerated in a multi-resolution transform of a luminance signal andcolor difference signals are preserved, the condition of a photographicobject is judged with reference to one or more of said luminancelow-frequency signals, color difference low-frequency signals preserved,and a luminance high-frequency signal, and the condition of a coringprocessing is changed on the basis of the result of the judgement.

[0082] Further, it is desirable that an image processing apparatus ofthe second and fifth of this invention further comprises means forpreserving the low-frequency signals of each level generated in amulti-resolution transform of a luminance signal and color differencesignals, means for judging the condition of a photographic object withreference to one or more of said luminance low-frequency signals, colordifference low-frequency signals preserved, and a luminancehigh-frequency signal, and means for changing the condition of a coringprocessing on the basis of the result of the judgement.

[0083] Further, it is desirable that a program of the third and sixth ofthis invention makes a computer function further as means for preservingthe low-frequency signals of each level generated in a multi-resolutiontransform of a luminance signal and color difference signals, means forjudging the condition of a photographic object with reference to one ormore of said luminance low-frequency signals, color differencelow-frequency signals preserved, and a luminance high-frequency signal,and means for varying the condition of a coring processing on the basisof the result of the judgement.

[0084] The above-mentioned will be explained in more detail. It is wellknown that granular noises are especially noticeable in a particularphotographic object such as the human face, skin, or the blue sky. Itwill be explained a case taken for example where a set of color imagesignals representing an original image are transformed into a luminancesignal Y and color difference signals I and Q of YIQ basis, and each ofthem undergoes a multi-resolution transform based on a wavelettransform. As described in the above, in determining the coringprocessing condition of the signal value at the coordinates (x, y) ofthe high-frequency component of the level n Wd_(n) of the luminancesignal Y, if the signal value at the coordinates (x/2^(i), y/2^(i)) ofthe low-frequency component of the level (n+i) S_(n+i) of the luminancesignal Y is referred to, the reference signal value indicates theaverage luminance value over the (2^(i)×2^(i)) pixels in theneighborhood of the coordinates (x, y) of the high-frequency componentof the level n Wd_(n) which is the object of the coring processing.

[0085] Further, by referring to the signal value at the coordinates(x/2^(i), y/2^(i)) of the low-frequency component of the level (n+i)S_(n+i) of the color difference signal I and the signal value at thecoordinates (x/2^(i), y/2^(i)) of the low-frequency component of thelevel (n+i) S_(n+i) of the color difference signal Q, it is possible toknow what kind of color as viewed widely has the area of the(2^(i)×2^(i)) pixels in the neighborhood of the coordinates of thehigh-frequency component of the level n Wd_(n) which is the object ofthe coring processing. In the case where this color is included in aparticular region such as the color of the human skin, or the color ofthe sky, there is the possibility that this region of color representsthe particular photographic object such as the human face or skin, orthe blue sky.

[0086] Further, by referring to the signal value at the coordinates(x/2^(i), y/2^(i)) of the high-frequency component of the level (n+i)Wh_(n+i), Wv_(n+i), and Wd_(n+i) of the luminance signal Y, it ispossible to obtain the information on to what degree a complex structureas viewed widely has the area of the (2^(i)×2^(i)) pixels in theneighborhood of the coordinates (x, y) of the high-frequency componentof the level n Wd_(n) which is the object of the coring processing.

[0087] By referring to this degree of complexity, it is possible todiscriminate the photographic objects having a similar hue such as awall painted in beige, the skin of a person, a piece of beige-coloredcloth. In the case where a photographic object is judged to be aparticular one such as the face or the skin of a person, or the blue skyby the synthetic judgement of the above-mentioned, by switching over thecoring condition of the signal value at the coordinates (x, y) of thehigh-frequency component of the level n Wd_(n) which is the object ofthe coring processing to a coring condition determined for theparticular photographic object, it becomes possible to apply preciselythe processing in the desired condition to a particular photographicobject such as the face, or the skin of a person, or the blue sky only.For the signal of a level higher than n to be referred to for thepurpose of the above-mentioned judgement, it is desirable to use alow-frequency signal of a level higher than that of the high-frequencysignal which is the object of the coring processing by 1 to 5 levels.That is, it is desirable to make i satisfy the inequality 1≦i≦5. Thenumber of the levels to be referred to may be one, but it is moredesirable to practice the judgement with reference to the value ofsignals of a plurality of levels and on the basis of the weightedaverage of the signal values.

[0088] In an image processing method of the first and fourth of thisinvention, it is desirable to apply a gradation transform to theluminance low-frequency image signal of the highest level before theapplication of an inverse multi-resolution transform.

[0089] Further, an image processing apparatus of the second and fifth ofthis invention further comprises means for applying a gradationtransform to the luminance low-frequency image signal of the highestlevel before the application of an inverse multi-resolution transform.

[0090] Further, it is desirable that a program of the third and sixth ofthis invention makes a computer function as means for applying agradation transform to the luminance low-frequency image signal of thehighest level before the application of an inverse multi-resolutiontransform.

[0091] The above-mentioned will be explained in more detail. Generallyspeaking, the range of the luminance distribution of an object scene ina-photographic image extends over a wide range of 10³ or 10⁴ in anarbitrary unit, but the luminance range which can be displayed by adisplay device or a print is only an order of 10². Therefore, becausethe wide luminance distribution of a scene of a photographic object iscompressed to a luminance range which can be displayed in producing animage signal, an operation to compress the gradation in the both sidesof high luminance and low luminance is practiced in most cases.

[0092] In a suitable mode of practice of this invention, for the imagesignal representing an original image in this invention, it is supposedto use a signal after the above-mentioned compression processing is onceapplied to it; however, sometimes it is generated a requirement suchthat a fine adjustment of the gradation should be carried out at thesame time while finishing operations of an image such as the suppressionof a granular noise and the enhancement of sharpness are being carriedout. In this case, if it is tried to do all of the compressionprocessing of gradation and the calculations relating to this inventionover again, the calculation takes a considerable time which lowers theproductivity remarkably. However, if the fine adjustment of gradation ismade for the luminance low-frequency component of the highest level NS_(N) generated by a multi-resolution transform of a luminance signal ofthis invention, the same effect can be obtained by the re-calculation ofan inverse multi-resolution transform of the luminance signal only.

[0093] In an image processing method of the first and fourth of thisinvention, it is desirable to apply a gray-balance adjustment to thecolor difference low-frequency image signals of the highest level beforethe application of an inverse multi-resolution transform.

[0094] Further, it is desirable that an image processing apparatus ofthe second and fifth of this invention further comprises means forapplying a gray-balance adjustment to the color difference low-frequencyimage signals of the highest level before the application of an inversemulti-resolution transform.

[0095] Further, it is desirable that a program of the third and sixth ofthis invention makes a computer function further as means for applying agray-balance adjustment to the color difference low-frequency imagesignal of the highest level before the application of an inversemulti-resolution transform.

[0096] The above-mentioned will be explained in more detail. Because thedensity of a usual color negative film is adjusted by the amount ofexposure at the time of printing, it is designed in such a way that theB, G, and R densities to be produced by a gray light exposure have acertain difference between them. Further, as regards not only a colornegative film but also a reversal film or an image photographed by adigital still camera or the like, the balance among the B, G, R signalsvaries due to the difference in the color temperature between thephotographing light sources such as the day-light and a fluorescentlight and the difference in the spectral sensitivity between thephotographing device and the human eye. For this reason, it is necessaryto carry out a gray balance adjustment in order that a feeling of thebalance of the R, G, and B signal intensities being proper may beobtained when the image is observed.

[0097] In a suitable mode of practice of this invention, for the imagesignal representing an original image in this invention, it is assumedto use a signal after the above-mentioned gray balance adjustment hasbeen once applied to it; however, in actual operations, sometimes it isgenerated a requirement that the fine adjustment of a gray balanceshould be carried out at the same time while finishing operations of animage such as the suppression of a granular noise and the enhancement ofsharpness are being carried out. In this case, if it is tried to do allof the gray balance adjustment operation and the calculations relatingto this invention over again, the calculation takes a considerable timewhich lowers the productivity remarkably. However, if the fineadjustment of a gray balance is carried out for the color differencelow-frequency components of the highest level N S_(N)'s generated by amulti-resolution transform of color difference signals of thisinvention, the same effect can be obtained by the re-calculation of aninverse multi-resolution transform of the color difference signals only.In this way, this invention provides a high-speed means for adjusting agray balance.

BRIEF DESCRIPTION OF THE DRAWINGS

[0098]FIG. 1 is a drawing for explaining a mode of transform of onelevel in the case where a wavelet transform is used as amulti-resolution transform;

[0099]FIG. 2 is a drawing for explaining a mode of inverse transform ofone level in the case where a wavelet inverse transform is used as aninverse multi-resolution transform;

[0100]FIG. 3 is a drawing showing the concept of a multi-resolutiontransform and an inverse multi-resolution transform at multiple levels;

[0101]FIG. 4(a) to FIG. 4(c) are drawings each showing an example of acoring processing;

[0102]FIG. 5 is a drawing showing a flow of a processing for obtaining aluminance signal of enhanced sharpness;

[0103]FIG. 6 is a system block diagram of a photographic film imagereading apparatus with an image processing apparatus of this inventionbuilt in;

[0104]FIG. 7 is a system block diagram of a mode of practice of aninternal processing of an image processing apparatus of this invention;

[0105]FIG. 8 is a system block diagram of another mode of practice of aninternal processing of an image processing apparatus of this invention;

[0106]FIG. 9 is a drawing showing a mode of practice of a sharpnessenhancement processing shown in FIG. 8;

[0107]FIG. 10 is a drawing showing a condition of a coring processing;

[0108]FIG. 11 is a drawing showing a condition of a coring processing;

[0109]FIG. 12 is a drawing showing a condition of a coring processing;

[0110]FIG. 13 is a drawing showing a condition of a coring processing;

[0111]FIG. 14 is a drawing showing an example of changing a condition ofa coring processing;

[0112]FIG. 15 is a system block diagram of another mode of practice ofan internal processing of an image processing apparatus of thisinvention;

[0113]FIG. 16 is a drawing showing an example of setting to change thecondition of a coring processing;

[0114]FIG. 17 is a drawing showing an example of setting to change thecondition of a coring processing;

[0115]FIG. 18 is a drawing showing an example of setting to change thecondition of a coring processing;

[0116]FIG. 19 is a drawing showing an example of setting to change thecondition of a coring processing;

[0117]FIG. 20 is a drawing showing the result of the modification of thecondition of a coring processing;

[0118]FIG. 21 is a system block diagram of further another mode ofpractice of an internal processing of an image processing apparatus ofthis invention; and

[0119]FIG. 22 is a system block diagram of furthermore another mode ofpractice of an internal processing of an image processing apparatus ofthis invention.

BEST MODE FOR CARRYING OUT THE INVENTION

[0120] In the following, with reference to the drawings, concreteexamples of the embodiment of an image processing method and an imageprocessing apparatus will be explained.

[0121]FIG. 6 is a system block diagram of a photographic film imagereading apparatus with an image processing apparatus of this inventionbuilt in. An image on a color negative film or a color reversal film isphotoelectrically converted into a set of transmission light quantitysignals by means of a film scanning apparatus known to the public with alight receiving element such as a line CCD sensor or an area CCD sensor.The set of transmission light quantity signals obtained are amplified byan amplifier, and are converted into a set of digital signals by an A/Dconversion device. Subsequently, corrections for removing noises properto a light receiving element such as a dark fixed pattern noisecorrection and a shading correction are practiced, and further, acalibration processing for correcting the individual difference of theapparatus caused by the dispersion of the characteristics of a lightreceiving sensor, a color separation filter, a light source lamp, alens, and other optical parts is carried out.

[0122] Subsequently, the set of transmission light quantity signals,having undergone the correction, are transformed into a set of densitysignals by logarithmic transform or the like. This set of densitysignals are sent to a processing condition judgement section 61, whereprocessing conditions for practicing image processing are calculated,and on the basis of these processing conditions, the above-mentioned setof density signals are transformed into a set of color image signalswhich have already undergone a color balance correction processing, agradation adjustment processing, a negative-to-positive reversingprocessing. The result of this is displayed on an image display device62, and also it is possible to carry out a re-processing by changing theconditions of color balance correction, gradation correction, andnegative-to-positive reversing.

[0123] The fixedly determined set of color image signals to which theapplication of a color balance correction, a gradation correction, and anegative-to-positive reversing has already been finished are sent to animage processing apparatus 63 of this invention, and transformed into aset of processed image signals on the basis of a condition instructed bythe processing condition judgement section or an operator. Also it ispossible that, in this stage, the image is again displayed on the imagedisplay device 62, and an operator changes the processing condition andorders re-processing if he judges it to be necessary. The processed setof image signals which have been fixedly determined are further sent toone of output adjustment sections 64 corresponding to the respectiveoutput apparatus, and after a transform of color space, a resolutionadjustment, a reduction or enlargement processing, a compressionprocessing, a data format transform, etc. are carried out, the set ofcolor image signals are outputted to a printer, recorded in a media, ortransmitted through a communication line.

[0124]FIG. 7 is an example of a system block diagram of the internalprocessing of the image processing apparatus 63 shown in FIG. 6. Aninputted set of R, G, and B signals are transformed into a set of Y, I,and Q signals. The I signal and the Q signal are separately subjected toa multi-resolution transform, and after their high-frequency componentsare suppressed, the multi-resolution signals are transformed into an I′signal and a Q′ signal respectively by an inverse multi-resolutiontransform. The Y signal undergoes a multi-resolution transform, andafter the high-frequency components are suppressed, the multi-resolutionsignals are converted into a Y′ signal by an inverse multi-resolutiontransform. Finally, the set of the signals of I′, Q′ and Y′ aretransformed into a set of signals of RGB basis, to be outputted as a setof signals of R′, G′, and B′ after the processing.

[0125]FIG. 8 is a system block diagram of another example of theinternal processing of the image processing apparatus shown in FIG. 6.In this system, Y signal undergoes a multi-resolution transform, andafter the high-frequency components are suppressed, the multi-resolutionsignals undergo an inverse multi-resolution transform, and by thefurther application of a sharpness enhancement processing, a Y′ signalis obtained.

[0126]FIG. 9 shows an example of a mode of practice of theabove-mentioned sharpness enhancement processing. An inputted signal Inpasses a two-dimensional low pass filter LPF, and a low-frequencycomponent L is abstracted. Next, by taking the difference between theinput signal In and the low-frequency component L, a high-frequencycomponent H is abstracted. This high-frequency component H istransformed into H′ by an amplification processing, and by the additionof this amplified high-frequency component H′ to the low-frequencycomponent L, an output signal Out is generated.

[0127]FIG. 10 to FIG. 13 are graphs each showing a condition of a coringprocessing practiced in FIG. 7 and FIG. 8. FIG. 10 shows a coringcondition for the high-frequency components of the level 1, and FIG. 11,FIG. 12, and FIG. 13 correspond to the level 2, level 3, and level 4respectively. The x-axis in each graph represents the absolute value ofan input signal before the coring processing, and the y-axis representsthe absolute value of the output signal after the coring processing. Forthe ease of understanding, the values in the x-axis and y-axis arenormalized by the standard deviation σ of the total input signalabsolute values of the corresponding component. For example, the pointwhere the value in respect of the x-axis is 1.96 corresponds to thepoint at 5% to the total as counted from the largest one in the casewhere the total signal values of the corresponding component arearranged in the order of the absolute value from the maximum to thesmaller ones. Besides, in the case where the input signal is negative,by the use of the absolute value in the graph shown in any one of FIG.10 to FIG. 13, the absolute value of the output signal is obtained, andby attaching a minus sign to it, an output value is obtained.

[0128]FIG. 14 shows an example in which a coring condition is changedwith reference to the high-frequency component of the level (n+1) whichis higher than the level n that is the object of the coring processing,with the coring condition shown in FIG. 10 to FIG. 13 taken as thebasis. The x-axis of FIG. 14 represents the absolute value of the signalvalue at the corresponding coordinate of the correspondinghigh-frequency component of the level (n+1) which is higher than thelevel n that is the object of the coring processing. For the ease ofunderstanding, this value is normalized on the basis of the standarddeviation σ of the signal absolute value of the correspondinghigh-frequency component of the level (n+1). The value in respect of they-axis represents the rate of change when the coring condition of thelevel n is changed with reference to the value of the level (n+1).

[0129] In the following, explanation will be given with it taken forinstance a case where a multi-resolution transform is practiced by theuse of a wavelet transform. It is assumed that the signal value at thecoordinates (x, y) of the high-frequency component of the level 1 YWv₁is 1.2 times of the standard deviation of the total signal values of theYWv₁. Because the value in respect of the y-axis corresponding to thepoint at the value 1.2 in respect of the x-axis is 0.5 in FIG. 10, thebasic output value at the coordinates (x, y) of the high-frequencycomponent of the level 1 YWv₁ is 0.5 time of the above-mentionedstandard deviation. Now, it is assumed that the signal value at thecoordinates (x/2, y/2) of the high-frequency component of the level 2YWv₂ is 1.4 times of the standard deviation of the total signal valuesof the YWv₂. Because the value in respect of the y-axis corresponding tothe point at the value 1.4 in respect of the x-axis is 0.5 in FIG. 14,the above-mentioned basic output value is adjusted by the rate of change0.5. That is, (0.5−1.2)×0.5+1.2=0.85 is the signal value after thecoring processing at the coordinates (x, y) of the high-frequencycomponent of the level 1 YWv₁.

[0130]FIG. 15 shows a system block diagram of another mode of theinternal processing of the image processing apparatus shown in FIG. 6.In this system, the low-frequency components of every level YS₁ toYS_(N) are preserved in an multi-resolution transform of the Y signal,and in a coring processing of the high-frequency components of eachlevel of the Y signal, the above-mentioned low-frequency components ofevery level YS₁ to YS_(N) are referred to. FIG. 16 to FIG. 19 showexamples of setting for changing the condition of a coring processingwith reference to the low-frequency components YS₁ to YS_(N).

[0131]FIG. 16 to FIG. 19 represent the settings of the level 1 to 4respectively. The x-axis represents the signal value (expressed in 8bits) of a low-frequency component, and the y-axis represents the cutoffvalue of the corresponding coring condition as normalized on the basisof the standard deviation. In the case where the absolute value of thesignal value which is the object of the coring is not greater than thecutoff value, the signal value after the coring processing is made zero.

[0132] In the following, explanation will be given with it taken forinstance a case where a multi-resolution transform is practiced by theuse of a wavelet transform. It is assumed that the basic coringconditions of the high-frequency components of the level 1 to level 4correspond to those shown in FIG. 10 to FIG. 13 respectively. Forexample, in applying a coring processing to the signal value at thecoordinates (x, y) of the high-frequency component of the level 1 YWv₁,the signal value at the coordinates (x, y) of the correspondinglow-frequency component of the level 1 YS₁ is referred to. If thissignal value referred to is 210, the corresponding cutoff value is 0.4in FIG. 16. On the other hand, the cutoff value of the basic coringcondition of the high-frequency component of the level 1 shown in FIG.10 is 0.7. Therefore, by multiplying the amount of shift of the curve ofthe basic coring condition from the straight line y=x by 0.4/0.7, thecoring condition is modified to have a shape such that the cutoff valuebecomes 0.4.

[0133] The result of this modification is shown in FIG. 20. The dottedline represents the basic coring condition, and the solid linerepresents the modified coring condition. If the signal value at thecoordinates (x, y) of YWv₁ which is the object of the coring is 1.2times of the standard deviation of the total values of YWv₁, the valuein respect of the y-axis corresponding to the value 1.2 in respect ofthe x-axis in the modified coring condition shown in FIG. 20 is 0.8;therefore, the signal value after the coring is determined to be a valueof the standard deviation multiplied by 0.8. In the above-mentionedexample, the signal value of a low-frequency component of the same levelas the high-frequency component which is the object of the coring isreferred to, but this invention is not limited to this; for example, inapplying a coring processing to the signal value at the coordinates (x,y) of the high-frequency component of the level 1 YWv₁, even byreferring to the signal value at the coordinates (x/2, y/2) of thelow-frequency component of the level 2 YS₂, the coring condition iscontrolled in the same way.

[0134]FIG. 21 is a system block diagram of another mode of the internalprocessing of the image processing apparatus shown in FIG. 6. In thissystem, in applying a multi-resolution transform to Y, I, and Q signals,all of the low-frequency components of every level YS₁ to YS_(n), IS₁ toIS_(n), and QS₁ to QS_(n) are preserved, and the signal values of thoseare referred to in a coring condition setting section. In the coringcondition setting section, the values of the Y, I, and Q signalscorresponding to the coordinates of the object of the coring arereferred to, and from the hue and range of luminance of the signals, itis judged whether or not the object is a particular photographic objectsuch as the face or the skin of a person, or the blue sky, and the basiccoring condition is selected on the basis of the result of thejudgement. Further, also the reference to a high-frequency component ora low-frequency component of a level higher than the object of thecoring is carried out, the basic coring condition is changed by themethod explained in the above, and the coring condition to be applied isdetermined and transmitted to a coring processing section. Following thecondition transmitted, the coring processing section practices a coringprocessing of the luminance high-frequency components.

[0135]FIG. 22 is a system block diagram of further another mode of theinternal processing of the image processing apparatus shown in FIG. 6.In this system, the low-frequency components of Y signal are used in aninverse multi-resolution transform after they have undergone a gradationadjustment by a gradation transforming means. Further, the low-frequencycomponents of I and Q signals are used in an inverse multi-resolutiontransform after they have been adjusted by a gray balance adjustingmeans. In this mode of the invention, by controlling these gradationtransforming means and gray balance adjusting means, it is possible tocarry out a gradation adjustment and a gray balance adjustment rapidlyby calculating again only the inverse multi-resolution transform.

[0136] In this invention, because a set of color image signalsrepresenting an original image is transformed into a luminance signaland color difference signals, a multi-resolution transform is applied tothe color difference signals to transform them into color differencemulti-resolution signals, and the high-frequency signals of the level 1of said color difference multi-resolution signals are suppressed, anoise looking like colors being out of registration is effectivelyremoved. On top of it, because suppression is not equally applied to theluminance multi-resolution signals, the degradation of sharpnessaccompanied by the removal of a noise looking like colors being out ofregistration can be prevented.

[0137] Further, by the application of a coring processing using acondition different for each level from other levels to thehigh-frequency signals of each level of the luminance multi-resolutionsignals, a granular noise is remarkably removed. This granular noiseremoval process is different from the method disclosed in thepublication of the unexamined patent application H9-22460 in which aparticular frequency band is equally suppressed, but is designed toselectively control only an area where a granular noise is present;therefore, an appearance of blur is not produced in the periphery of thebridge of the nose and around the eyes. In addition to theabove-mentioned, because the amount of calculation required for thisinvention is much less than a technology in which a process forproducing a blurred image by means of a large-sized two-dimensionalfilter is necessary, the processing time is shortened remarkably.

[0138] Further, by an effective utilization of the information on amulti-resolution transform, it is possible to effectively control animage processing condition through a wide-view analysis of an imagewithout producing a new calculation load. As the result of that, on thebasis of the above-mentioned situation, this invention is capable ofsuppressing a granular noise signal included in a color image signal aswell as enhancing the sharpness of an image, while preserving the shadein the periphery of the bridge of the noise and that around the eyes,without producing a noise looking like colors being out of registrationand a smooth expressionless makeup face, without producing a noiselooking as if fine powders are scattered, and without producing afalse-color contour in the vicinity of the edge or false-color spots inthe flat portion.

1. An image processing method, comprising: transforming a set of colorimage signals representing an original image into a luminance signal andcolor difference signals; transforming the luminance signal and thecolor difference signals separately into multi-resolution signals of thelevel 1 to the level N to make them luminance multi-resolution signalsand color difference multi-resolution signals; applying an inversemulti-resolution transform to the color difference multi-resolutionsignals to make them processed color difference signals, aftersuppressing the high-frequency components of the level 1 of the colordifference multi-resolution signals; applying an inversemulti-resolution transform processing to the luminance multi-resolutionsignals to make them a processed luminance signal, after applying acoring processing using a condition for each level different from otherlevels to the high-frequency signals of each level of the luminancemulti-resolution signals; transforming the processed luminance signaland the processed color difference signals into a set of processed colorimage signals.
 2. An image processing method, comprising: transforming aset of color image signals representing an original image into aluminance signal and color difference signals; transforming theluminance signal and the color difference signals separately intomulti-resolution signals of the level 1 to the level N to make themluminance multi-resolution signals and color difference multi-resolutionsignals; applying an inverse multi-resolution transform to the colordifference multi-resolution signals to make them processed colordifference signals, after suppressing the high-frequency components ofthe level 1 of the color difference multi-resolution signals; applyingan inverse multi-resolution transform processing to the luminancemulti-resolution signals to make them a processed luminance signal,after applying a coring processing using a condition for each leveldifferent from other levels to the high-frequency signals of each levelof the luminance multi-resolution signals; applying a sharpnessenhancement processing to the processed luminance signal to make it aluminance signal of enhanced sharpness; transforming the luminancesignal of enhanced sharpness and the processed color difference signalsinto a set of processed image signals.
 3. The image processing method ofclaim 1, wherein the set of color image signals representing an originalimage are digital image signals obtained by the scanning of a dye imageformed on a silver halide film.
 4. The image processing method of claim2, wherein the set of color image signals representing an original imageare digital image signals obtained by the scanning of a dye image formedon a silver halide film.
 5. The image processing method of claim 1,wherein the coring processing to the luminance multi-resolution signalsincludes at least one processing which is practiced under a non-linearcoring condition such that the rate of change for the signal values ofthe higher rank corresponding to at least 5% to the total number ofsignal values with respect to the absolute value of an input signalbecomes smaller than 10%, and for the signal values of the lower rankcorresponding to at least 15% to the total number become substantiallyzero.
 6. The image processing method of claim 2, wherein the coringprocessing to the luminance multi-resolution signals includes at leastone processing which is practiced under a non-linear coring conditionsuch that the rate of change for the signal values of the higher rankcorresponding to at least 5% to the total number of signal values withrespect to the absolute value of an input signal becomes smaller than10%, and for the signal values of the lower rank corresponding to atleast 15% to the total number become substantially zero.
 7. The imageprocessing method of claim 1, wherein the condition for the coringprocessing of the luminance multi-resolution signals of the level n ischanged with reference to a high-frequency signal of a higher level thann.
 8. The image processing method of claim 2, wherein the condition forthe coring processing of the luminance multi-resolution signals of thelevel n is changed with reference to a high-frequency signal of a higherlevel than n.
 9. The image processing method of claim 1, wherein thecondition for the coring processing of a luminance multi-resolutionsignal of the level n is such one as to preserve the low-frequencysignal of each level generated in a multi-resolution transform, and tobe changed with reference to the low-frequency signal of the level n.10. The image processing method of claim 2, wherein the condition forthe coring processing of a luminance multi-resolution signal of thelevel n is such one as to preserve the low-frequency signal of eachlevel generated in a multi-resolution transform, and to be changed withreference to the low-frequency signal of the level n.
 11. The imageprocessing method of claim 1, wherein the condition for the coringprocessing of a luminance multi-resolution signal of the level n is suchone as to preserve the low-frequency signal of each level generated inthe multi-resolution transform, and to be changed with reference to alow-frequency signal of a level higher than n.
 12. The image processingmethod of claim 2, wherein the condition for the coring processing of aluminance multi-resolution signal of the level n is such one as topreserve the low-frequency signal of each level generated in themulti-resolution transform, and to be changed with reference to alow-frequency signal of a level higher than n.
 13. The image processingmethod of claim 1, wherein luminance low-frequency signals and colordifference low-frequency signals of each level generated in amulti-resolution transform of the luminance signal and the colordifference signals are preserved; the condition of a photographic objectis judged with reference to one or more of the luminance low-frequencysignals, the color difference low-frequency signals, and a luminancehigh-frequency signal; and the condition of the coring processing ischanged on the basis of the result of the judgement.
 14. The imageprocessing method of claim 2, wherein luminance low-frequency signalsand color difference low-frequency signals of each level generated in amulti-resolution transform of the luminance signal and the colordifference signals are preserved; the condition of a photographic objectis judged with reference to one or more of the luminance low-frequencysignals, the color difference low-frequency signals, and a luminancehigh-frequency signal; and the condition of the coring processing ischanged on the basis of the result of the judgement.
 15. The imageprocessing method of claim 1, further comprising the step of applying agradation transform to the luminance low-frequency image signal of thehighest level, before applying the inverse multi-resolution transformprocessing.
 16. The image processing method of claim 2, furthercomprising the step of applying a gradation transform to the luminancelow-frequency image signal of the highest level, before applying theinverse multi-resolution transform processing.
 17. The image processingmethod of claim 1, further comprising the step of applying agray-balance adjustment to the color difference low-frequency imagesignals of the highest level, before applying the inversemulti-resolution transform.
 18. The image processing method of claim 2,further comprising the step of applying a gray-balance adjustment to thecolor difference low-frequency image signals of the highest level,before applying the inverse multi-resolution transform.
 19. An imageprocessing apparatus comprising: a first transforming device fortransforming a set of color image signals, representing an originalimage, into a luminance signal and color difference signals; a secondtransforming device for transforming the luminance signal and the colordifference signals separately into multi-resolution signals of the level1 to the level N to make them luminance multi-resolution signals andcolor difference multi-resolution signals; a third transforming devicefor applying an inverse multi-resolution transform to the colordifference multi-resolution signals to transform them into processedcolor difference signals, after suppressing high-frequency signals ofthe level 1 of said color difference multi-resolution signals; a coringprocessing device for applying a coring processing using a condition foreach level different from other levels to high-frequency signals of eachlevel of the luminance multi-resolution signals; a fourth transformingdevice for applying an inverse multi-resolution transform to theluminance multi-resolution signals, having already undergone the coringprocessing, to transform them into a processed luminance signal; and afifth transforming device for transforming the processed luminancesignal and the processed color difference signals into a set ofprocessed color image signals.
 20. An image processing apparatuscomprising: a first transforming device for transforming a set of colorimage signals representing an original image into a luminance signal andcolor difference signals; a second transforming device for transformingthe luminance signal and the color difference signals separately intomulti-resolution signals of the level 1 to the level N to make themluminance multi-resolution signals and color difference multi-resolutionsignals; a third transforming device for applying an inversemulti-resolution transform to the color difference multi-resolutionsignals to transform them into processed color difference signals, aftersuppressing high-frequency signals of the level 1 of said colordifference multi-resolution signals; a coring processing device forapplying a coring processing using a condition for each level differentfrom other levels to high-frequency signals of each level of theluminance multi-resolution signals; a fourth transforming device forapplying an inverse multi-resolution transform to the luminancemulti-resolution signals, having already undergone the coringprocessing, to transform them into a processed luminance signal; asharpness enhancing device for applying a sharpness enhancementprocessing to the processed luminance signal to make it a luminancesignal of enhanced sharpness; and a fifth transforming device fortransforming the luminance signal of enhanced sharpness and theprocessed color difference signals into a set of processed color imagesignals.
 21. The image processing apparatus of claim 19, wherein the setof color image signals representing an original image are digital imagesignals obtained by the scanning of a dye image formed on a silverhalide film.
 22. The image processing apparatus of claim 20, wherein theset of color image signals representing an original image are digitalimage signals obtained by the scanning of a dye image formed on a silverhalide film.
 23. The image processing apparatus of claim 19, wherein thecoring processing device includes at least one processing means whichperforms under a non-linear coring condition such that the rate ofchange for the signal values of the higher rank corresponding to atleast 5% to the total number of signal values with respect to theabsolute value of an input signal becomes smaller than 10%, and for thesignal values of the lower rank corresponding to at least 15% to thetotal number become substantially zero.
 24. The image processingapparatus of claim 20, wherein the coring processing device includes atleast one processing means which performs under a non-linear coringcondition such that the rate of change for the signal values of thehigher rank corresponding to at least 5% to the total number of signalvalues with respect to the absolute value of an input signal becomessmaller than 10%, and for the signal values of the lower rankcorresponding to at least 15% to the total number become substantiallyzero.
 25. The image processing apparatus of claim 19, wherein thecondition for the coring processing of the luminance multi-resolutionsignals of the level n is changed with reference to a high-frequencysignal of a higher level than n.
 26. The image processing apparatus ofclaim 20, wherein the condition for the coring processing of theluminance multi-resolution signals of the level n is changed withreference to a high-frequency signal of a higher level than n.
 27. Theimage processing apparatus of claim 19, wherein the condition for thecoring processing of a luminance multi-resolution signal of the level nis such one as to preserve the low-frequency signal of each levelgenerated in a multi-resolution transform, and to be changed withreference to the low-frequency signal of the level n.
 28. The imageprocessing apparatus of claim 20, wherein the condition for the coringprocessing of a luminance multi-resolution signal of the level n is suchone as to preserve the low-frequency signal of each level generated in amulti-resolution transform, and to be changed with reference to thelow-frequency signal of the level n.
 29. The image processing apparatusof claim 19, wherein the condition for the coring processing of aluminance multi-resolution signal of the level n is such one as topreserve the low-frequency signal of each level generated in themulti-resolution transform, and to be changed with reference to alow-frequency signal of a level higher than n.
 30. The image processingapparatus of claim 20, wherein the condition for the coring processingof a luminance multi-resolution signal of the level n is such one as topreserve the low-frequency signal of each level generated in themulti-resolution transform, and to be changed with reference to alow-frequency signal of a level higher than n.
 31. The image processingapparatus of claim 19, further comprising: a preserving device forpreserving luminance low-frequency signals and color differencelow-frequency signals of each level generated in a multi-resolutiontransform of the luminance signal and the color difference signals; ajudgment device for judging the condition of a photographic object withreference to one or more of the luminance low-frequency signals, thecolor difference low-frequency signals, and a luminance high-frequencysignal; and a condition changing device for changing the condition ofthe coring processing on the basis of the result of the judgment. 32.The image processing apparatus of claim 20, further comprising: apreserving device for preserving luminance low-frequency signals andcolor difference low-frequency signals of each level generated in amulti-resolution transform of the luminance signal and the colordifference signals; a judgment device for judging the condition of aphotographic object with reference to one or more of the luminancelow-frequency signals, the color difference low-frequency signals, and aluminance high-frequency signal; and a condition changing device forchanging the condition of the coring processing on the basis of theresult of the judgment.
 33. The image processing apparatus of claim 19,further comprises a gradation transforming device for applying agradation transform to the luminance low-frequency image signal of thehighest level, before applying the inverse multi-resolution transform.34. The image processing apparatus of claim 20, further comprises agradation transforming device for applying a gradation transform to theluminance low-frequency image signal of the highest level, beforeapplying the inverse multi-resolution transform.
 35. The imageprocessing apparatus of claim 19, further comprises a gray-balanceadjusting device for applying a gray-balance adjustment to the colordifference low-frequency image signals of the highest level beforeapplying the inverse multi-resolution transform.
 36. The imageprocessing apparatus of claim 20, further comprises a gray-balanceadjusting device for applying a gray-balance adjustment to the colordifference low-frequency image signals of the highest level beforeapplying the inverse multi-resolution transform.
 37. A program formaking a computer function as an image processing apparatus, the imageforming apparatus comprising: a first transforming device fortransforming a set of color image signals, representing an originalimage, into a luminance signal and color difference signals; a secondtransforming device for transforming the luminance signal and the colordifference signals separately into multi-resolution signals of the level1 to the level N to make them luminance multi-resolution signals andcolor difference multi-resolution signals; a third transforming devicefor applying an inverse multi-resolution transform to the colordifference multi-resolution signals to transform them into processedcolor difference signals, after suppressing high-frequency signals ofthe level 1 of said color difference multi-resolution signals; a coringprocessing device for applying a coring processing using a condition foreach level different from other levels to high-frequency signals of eachlevel of the luminance multi-resolution signals; a fourth transformingdevice for applying an inverse multi-resolution transform to theluminance multi-resolution signals, having already undergone the coringprocessing, to transform them into a processed luminance signal; and afifth transforming device for transforming the processed luminancesignal and the processed color difference signals into a set ofprocessed color image signals.
 38. A program for making a computerfunction as an image processing apparatus, the image forming apparatuscomprising: a first transforming device for transforming a set of colorimage signals representing an original image into a luminance signal andcolor difference signals; a second transforming device for transformingthe luminance signal and the color difference signals separately intomulti-resolution signals of the level 1 to the level N to make themluminance multi-resolution signals and color difference multi-resolutionsignals; a third transforming device for applying an inversemulti-resolution transform to the color difference multi-resolutionsignals to transform them into processed color difference signals, aftersuppressing high-frequency signals of the level 1 of said colordifference multi-resolution signals; a coring processing device forapplying a coring processing using a condition for each level differentfrom other levels to high-frequency signals of each level of theluminance multi-resolution signals; a fourth transforming device forapplying an inverse multi-resolution transform to the luminancemulti-resolution signals, having already undergone the coringprocessing, to transform them into a processed luminance signal; asharpness enhancing device for applying a sharpness enhancementprocessing to the processed luminance signal to make it a luminancesignal of enhanced sharpness; and a fifth transforming device fortransforming the luminance signal of enhanced sharpness and theprocessed color difference signals into a set of processed color imagesignals.
 39. The program of claim 37, wherein the set of color imagesignals representing an original image are digital image signalsobtained by the scanning of a dye image formed on a silver halide film.40. The program of claim 38, wherein the set of color image signalsrepresenting an original image are digital image signals obtained by thescanning of a dye image formed on a silver halide film.
 41. The programof claim 34, wherein the coring processing device includes at least oneprocessing means which performs under a non-linear coring condition suchthat the rate of change for the signal values of the higher rankcorresponding to at least 5% to the total number of signal values withrespect to the absolute value of an input signal becomes smaller than10%, and for the signal values of the lower rank corresponding to atleast 15% to the total number become substantially zero.
 42. The programof claim 38, wherein the coring processing device includes at least oneprocessing means which performs under a non-linear coring condition suchthat the rate of change for the signal values of the higher rankcorresponding to at least 5% to the total number of signal values withrespect to the absolute value of an input signal becomes smaller than10%, and for the signal values of the lower rank corresponding to atleast 15% to the total number become substantially zero.
 43. The programof claim 37, wherein the condition for the coring processing of theluminance multi-resolution signals of the level n is changed withreference to a high-frequency signal of a higher level than n.
 44. Theprogram of claim 38, wherein the condition for the coring processing ofthe luminance multi-resolution signals of the level n is changed withreference to a high-frequency signal of a higher level than n.
 45. Theprogram of claim 37, wherein the condition for the coring processing ofa luminance multi-resolution signal of the level n is such one as topreserve the low-frequency signal of each level generated in amulti-resolution transform, and to be changed with reference to thelow-frequency signal of the level n.
 46. The program of claim 38,wherein the condition for the coring processing of a luminancemulti-resolution signal of the level n is such one as to preserve thelow-frequency signal of each level generated in a multi-resolutiontransform, and to be changed with reference to the low-frequency signalof the level n.
 47. The program of claim 37, wherein the condition forthe coring processing of a luminance multi-resolution signal of thelevel n is such one as to preserve the low-frequency signal of eachlevel generated in the multi-resolution transform, and to be changedwith reference to a low-frequency signal of a level higher than n. 48.The program of claim 38, wherein the condition for the coring processingof a luminance multi-resolution signal of the level n is such one as topreserve the low-frequency signal of each level generated in themulti-resolution transform, and to be changed with reference to alow-frequency signal of a level higher than n.
 49. A program of claim37, wherein the image processing apparatus further comprises: apreserving device for preserving luminance low-frequency signals andcolor difference low-frequency signals of each level generated in amulti-resolution transform of the luminance signal and the colordifference signals; a judgment device for judging the condition of aphotographic object with reference to one or more of the luminancelow-frequency signals, the color difference low-frequency signals, and aluminance high-frequency signal; and a condition changing device forchanging the condition of the coring processing on the basis of theresult of the judgment.
 50. A program of claim 38, wherein the imageprocessing apparatus further comprises: a preserving device forpreserving luminance low-frequency signals and color differencelow-frequency signals of each level generated in a multi-resolutiontransform of the luminance signal and the color difference signals; ajudgment device for judging the condition of a photographic object withreference to one or more of the luminance low-frequency signals, thecolor difference low-frequency signals, and a luminance high-frequencysignal; and a condition changing device for changing the condition ofthe coring processing on the basis of the result of the judgment. 51.The program of claim 37, wherein the image processing apparatus furthercomprises a gradation transforming device for applying a gradationtransform to the luminance low-frequency image signal of the highestlevel, before applying the inverse multi-resolution transform.
 52. Theprogram of claim 38, wherein the image processing apparatus furthercomprises a gradation transforming device for applying a gradationtransform to the luminance low-frequency image signal of the highestlevel, before applying the inverse multi-resolution transform.
 53. Theprogram of claim 37, wherein the image processing apparatus furthercomprises a gray-balance adjusting device for applying a gray-balanceadjustment to the color difference low-frequency image signals of thehighest level before applying the inverse multi-resolution transform.54. The program of claim 38, wherein the image processing apparatusfurther comprises a gray-balance adjusting device for applying agray-balance adjustment to the color difference low-frequency imagesignals of the highest level before applying the inversemulti-resolution transform.